Cis 530 Assignemnt 2

One important generalization to emerge from Table 1 is that the accuracy rate improves for each corpus using the tagger trained on both corpora instead of the taggers trained on a single corpus. This exemplifies that common result that accuracy improves by simply including more training data, even if it from a different genre. However, it is interesting to note that the accuracy improved more for the Brown corpus (1.5%) than for the WSJ (0.2%). This is presumably due to the fact that the sentences in the WSJ are more homogenous that those in the Brown corpus; thus, adding extra training data from a different genre provides less relevant information to the WSJ tagger, and improves the results less. Another generalization from Table 1 is that the performance on the WSJ test data is, on the whole, better than the performance on the Brown corpus. The accuracy of the WSJ tagger on the WSJ text was 96.0%, whereas the accuracy of the Brown tagger on the Brown corpus was 2.1% lower (93.9%). Furthermore, the WSJ tagger performed almost as well on the Brown corpus as the Brown tagger did (only 0.6% lower), while the Brown tagger performed 4.4% worse than the WSJ tagger did on the WSJ text. This pattern is probably due in part to the aforementioned homogeneity of the WSJ corpus in comparison to the Brown corpus. The WSJ corpus has only one genre, and thus a smaller variety of writing styles, syntactic constructions, and vocabulary. This means that less training data is needed to accurately predict the POS of a given word in the WSJ corpus. A further reason for the superior results on the WSJ text is probably the greater